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(54) Video coder, decoder and transmission system 



(57) A transmission system transmits a series of 
coded frames from a transmitting device to a receiving 
device, using both intra-trame coding and inter-frame 
coding. The receiving device decodes the frames and 
sends acknowledgment signals to the transmitting 
device. The transmitting device selects the reference 
frame employed in inter -frame coding according to 



these acknowledgment signals. The method of refer- 
ence frame selection can be varied according to an 
assessment of transmission channel quality. The 
assessment criteria, or the reference frame selection 
method itself, can be varied in response to input from a 
human user. 
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Description 

BACKGROUND OF THE INVENTION 

The present invention relates to a picture coder, 
picture decoder, and picture transmission system that 
combine good data compression performance with a 
high tolerance of frame dropouts. 

Recently there has been a proliferation of systems 
that transmit moving pictures through communication 
networks: examples include videophone, videoconfer- 
encing, and video-on-demand (VOD) systems. To 
reduce the volume of transmitted data, the pictures are 
digitized and compressively coded at the transmitting 
device, and decoded at the receiving device. 

Two basic types of moving-picture coding can be 
distinguished: intra-frame coding, and inter-frame cod- 
ing. Intra-frame coding codes each frame separately, or 
divides a frame into blocks and codes each block sepa- 
- rately Methods of intra-frame coding have been stand- 
ardized by, for example, the Joint Photographic Experts 
Group (JPEG). Intra-frame coding compresses the data 
by reducing spatial redundancy in each frame. 

Inter-frame coding reduces both spatial and tempo- 
ral redundancy, by coding only the differences between 
one frame and a preceding frame. Inter-frame coding 
may include motion compensation, which yields high 
data compression ratios. 

Methods that employ both intra- and inter-frame 
coding have been standardized by, for example, the 
Moving Picture Experts Group (MPEG) and the Tele- 
communication Standardization Sector of the Interna- 
tional Telecommunication Union (ITU-T). FIG. 21 
illustrates the coding system adopted in ITU-T Recom- 
mendation H.261. Intra-frame coding, indicated by 
hatching, is performed at regular intervals; inter-frame 
coding is carried out at other times. In inter-frame cod- 
ing, each frame is coded with reference to the immedi- 
ately preceding frame, as indicated by the arrows. 
Frames coded by inter-frame coding are in a sense pre- 
dicted from the preceding frames, and are referred to as 
P-frames, while frames coded by intra-frame coding are 
referred to as l-frames. In FIG. 21, frames a and i are I- 
frames, while frames, b to h, j, and k are P-frames. 

FIG. 22 shows an example of a problem that occurs 
in the H.261 scheme and similar schemes. If the receiv- 
ing device is unable to decode frame e for some reason, 
then frames f, g, and h will also be undecodable. The 
receiving device will have to wait until it receives the 
next l-frame (frame i) before decoding can resume. 
Dropout of a single frame can thus lead to lengthy and 
highly undesirable gaps in the received moving picture. 

Frame dropout can occur for a variety of reasons 
Frames may be dropped intentionally at the receiving 
end because, for example, the decoder has a slower 
processing speed than the coder and is unable to keep 
up. In networks that transmit data in packets or cells, 
packets or cells may be dropped en route when the net- 
work becomes overloaded. This can occur in local-area 



networks employing the well-known Ethernet system, 
for example, or in wide-area networks employing the 
well-known asynchronous transmission mode (ATM). 
Particularly in wide-area networks, packets or cells may 
5 also arrive out of sequence, because of having been 
transmitted over different network routes, for example; 
this again causes problems in the decoding of P- 
frames. 

To deal with these latter problems, some networks 

10 employ a protocol in which the transmitting device 
sends packets with attached serial numbers, and the 
receiving device rearranges the packets in the correct 
order, confirms their arrival, and sends requests for the 
retransmission of non-arriving packets back to the 

is transmitting device. A well-known example of this type 
of protocol is the Transmission Control Protocol (TCP). 

When network operation is unstable, however, and 
packets are dropped frequently, retransmission under 
this type of protocol can cause large cumulative delays 

20 tobuild up, which is unsuitable for the real-time trans- 
mission of moving pictures. When moving-picture data 
are transmitted, it is generally preferable to display new 
data, even if that means skipping a frame, rather than 
wait for the retransmission of old data. 

25 These problems are compounded in multi-point 
transmission schemes such as broadcasting and multi- 
casting schemes, which send the same data to multiple 
receiving sites. If the transmitting device heeds a 
retransmission request from one receiving site, it will 

30 often be forced to transmit to other sites a packet that 
those other sites have already received successfully, 
and the network load will be greatly increased. Broad- 
casting and multicasting are therefore usually carried 
out under a protocol that does not perform retransmis- 

35 sion, such as the User Datagram Protocol (UDP); but as 
a result, the probability of frame dropout increases. 

In wireless networks, frame dropout is a serious 
problem even when transmission takes place over a 
dedicated channel, instead of by packet or cell switch- 

40 ing. Wireless transmission is highly prone to error, and 
when the errors exceed the error-correcting capability of 
the receiving device, the usual practice is to discard a 
certain section of the data in order to re-establish valid 
data processing. Data dropouts therefore tend to be 

45 larger than in wireline networks. 

These factors limit the usefulness of the coding 
scheme illustrated in FIG. 21 to the transmission of 
moving pictures through telephone lines, integrated 
services digital networks (ISDNs), and other facilities 

so that offer a reliable link, equivalent to a physical circuit, 
between the transmitting and receiving devices. For 
transmission through other types of networks, in which 
frame dropout or skipping is to some extent unavoida- 
ble, the scheme illustrated in FIG. 23 is often employed: 

55 all frames are coded as l-frames, using JPEG coding, 
for example. 

When all frames are coded as l-frames, if a dropout 
occurs, as at frame e in FIG. 24, it has little effect on the 
perceived quality of the moving picture. In FIG. 24 only 
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frame e is lost; the succeeding frames f. g, and h can be 
decoded successfully, because their decoding does not 
depend on preceding frames. 

The problem with the all- l-frame scheme is that the 
data compression ratio is not very high, because tempo- 
ral redundancy is not removed. Much network band- 
width is therefore consumed. 

Japanese Patent Kokai Publication No. 95571/1995 
discloses an alternative scheme, illustrated in FIG. 25, 
in which P-frames b to h are all coded with reference to 
the preceding l-frame (a). Under this scheme, the loss 
of a P-frame does not affect the decoding of other P- 
frames. A disadvantage of this scheme is that the data 
compression ratio tends to decline with each succeed- 
ing P-frame, due to increasing temporal distance 
between the P-frame and the l-frame to which the P- 
frame is referenced. 



3 944 A2 4 

erence frame number, to enable the transmitting device 
to select a reference frame that the receiving device has 
successfully decoded. 

In a third preferred mode of operation, both nega- 
5 tive and positive acknowledgment signals are sent, the 
transmission channel quality is assessed according to 
these acknowledgment signals, and the method of ref- 
erence frame selection is varied according to the 
assessment. For example, the reference frame can be 
io selected as in the above first preferred mode under bad . 
channel conditions, and as in the above second pre- 
ferred mode under good channel conditions. 

The channel quality assessment criteria and refer- 
ence frame selection method can also be varied in 
is response to input from a human user. 

BRIEF DESCRIPTION OF THE DRAWINGS 



SUMMARY OF THE INVENTION 

One object of the present invention is, accordingly, 
to enable P-frames, including P-frames coded with ref- 
erence to a preceding P-frame, to be decoded after a 
frame dropout, without waiting for the next l-frame. 

Another object of the invention is to enable picture 
quality to adapt to transmission channel conditions. 

A further object is to enable data compression 
ratios to adapt to transmission channel conditions. 

Still another object is to provide a human user with 
control over the quality of transmitted moving pictures, 
in ways suitable for different transmission channel con- 
ditions. 

The invented transmission system transmits a 
series of frames from a transmitting device to a receiv- 
ing device. Intra-frame coding or inter-frame coding is 
selected for each frame, and the corresponding coding 
process is carried out at the transmitting device. When 
inter-frame coding is selected, the frame is coded with 
reference to a reference frame, the reference frame 
being a frame that was coded previously. The coded 
data resulting from intra-frame coding and inter-frame 
coding are transmitted to the receiving device. The 
receiving device decodes the decoded data, and sends 
acknowledgment signals back to the transmitting 
device. The transmitting device selects the reference 
frame on the basis of these acknowledgment signals. 

In a first preferred mode of operation, the receiving 
device transmits positive acknowledgment signals, and 
the transmitting device selects positively acknowledged 
frames as reference frames. 

In a second preferred mode of operation, the 
receiving device transmits negative acknowledgment 
signals, and the transmitting device selects the most 
recently coded frame as the reference frame, except 
when a negative acknowledgment signal is received. 
When a negative acknowledgment signal is received, 
the reference frame is set back to a frame preceding the 
negatively acknowledged frame. A negative acknowl- 
edgment signal may be accompanied by a desired ref- 



FIG. 1 is a block diagram of the moving-picture 
20 coder in the first embodiment. 

FIG. 2 is a block diagram of the moving -picture 
decoder in the first embodiment. 

FIG. 3 shows an example of inter-frame coding 
relationships in the first embodiment. 
25 FIG. 4 shows another example of inter-frame cod- 
ing relationships in the first embodiment. 

FIG. 5 shows an example of frame dropout in the 
first embodiment. 

FIG. 6 is a block diagram showing a variation of the 
30 moving-picture coder in the first embodiment. 

FIG. 7 is a block diagram of the reference frame 
updating, unit in the moving-picture coder in the second 
embodiment. 

FIG. 8 shows an example of frame dropout in the 
35 second embodiment. 

FIG. 9 shows an example of acknowledgment sig- 
nal dropout in the second embodiment. 

FIG. 10 is a block diagram of the reference frame 
updating unit in the moving -picture coder in the third 
40 embodiment. 

FIG. 1 1 shows an example of frame dropout in the 
third embodiment. 

FIG. 12 is a block diagram of the reference frame 
updating unit in the moving-picture coder in the fourth 
45 embodiment. 

FIG. 13 shows an example of frame dropout in the 
fourth embodiment. 

FIG. 14 shows an example of frame dropout fol- 
lowed by acknowledgment signal dropout in the fourth 
so embodiment. 

FIG. 15 is a block diagram of the moving-picture 
coder in the fifth embodiment. 

FIG. 16 is a block diagram of the moving -picture 
decoder in the fifth embodiment. 
55 FIG. 17 shows examples of frame dropout in the 
fifth embodiment 

FIG. 18 is a block diagram of the moving-picture 
coder in the sixth embodiment. 

FIG. 19 is a block diagram of the moving-picture 
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coder in the seventh embodiment 

FIG. 20 is a block diagram of the moving-picture 
decoder in the seventh embodiment. 

FIG. 21 illustrates inter-frame coding relationships 
in a first prior-art scheme 

FIG. 22 shows an example of frame dropout in the 
first prior-art scheme 

Fig. 23 illustrates intra-frame coding relationships in 
a second prior-art scheme 

FIG. 24 shows an example of frame dropout in the 
second prior-art scheme 

FIG. 25 illustrates inter-frame coding relationships 
in a third prior-art scheme 

DETAILED DESCRIPTION OF THE INVENTION 

Embodiments of the invention will be described with 
reference to the attached illustrative drawings. 

These embodiments can be realized as. for exam- 
ple, "semiconductor integrated clrcuitsT Circuit-level 
descriptions will be omitted, as methods of designing 
and fabricating the necessary integrated circuits are 
well known to those skilled in the art The invention can 
also be practiced through software means 

First embodiment 

The first embodiment illustrates the general config- 
uration of the coder and decoder in the invented mov- 
ing-picture transmission system, and shows one 
scheme for the selection of reference frames. Other 
schemes will be elaborated in later embodiments. 

FIG. 1 is a functional block diagram of the moving- 
picture coder 300 in the first embodiment. This coder, 
which is disposed in the transmitting device, comprises 
a moving-picture input unit 301, a coding unit 302, a 
decoding unit 303, a frame memory unit 304, a refer- 
ence frame memory unit 305. an intra/inter decision unit 
306, a coded data transmitting unit 307, an acknowledg- 
ment signal receiving unit 308. and a reference frame 
updating unit 309. 

The moving-picture input unit 301 furnishes mov- 
ing-picture data input from a video camera, for example, 
to the coding unit 302, frame by frame. The coding unit 
302 codes the input frame data, and furnishes the 
coded data to the decoding unit 303 and coded data 
transmitting unit 307. The coding unit 302 performs 
intra-frame coding and inter-frame coding at the direc- 
tion of the intra/inter decision unit 306. For inter-frame 
coding, the coding unit 302 refers to a reference frame 
stored in the reference frame memory unit 305. and 
codes only the difference between the current frame 
and the reference frame, preferably with motion com- 
pensation. 

The decoding unit 303 decodes the coded data 
received from the coding unit 302, and writes the 
decoded frames and their frame numbers into the frame 
memory unit 304. The frame memory unit 304 stores 
these decoded frames. The reference frame memory 



unit 305 stores one reference frame, which is copied 
from the frame memory 304 by the reference frame 
updating unit 309. 

The intra/inter decision unit 306 selects intra-frame 

5 coding or inter-frame coding and notifies the coding unit 
302, decoding unit 303, and coded data transmitting 
unit 307 of the selection. Normally, intra-frame coding is 
selected at regular intervals (once every thirty frames, 
for example), and inter-frame coding is selected at other 

10 times. The intra/inter decision unit 306 may receive a 
refresh signal (RFSH), however, which forces intra- 
frame coding to be selected. The refresh signal is sent 
from the decoder, as described later. When intra-frame 
coding is selected, the reference frame updating unit 

is 309 is also notified. 

The coded data transmitting unit 307 transmits the 
coded moving-picture data through a transmission 
channel (not visible) to a moving-picture decoder, or to 
a plurality of moving-picture decoders at different 

20 receiving sites. The coded data transmitting unit 307 
multiplexes header information, including a reference 
frame number (for P-frames), an intra/inter flag, and 
other necessary information such as transmission rout- 
ing information, onto the output data stream. 

25 The acknowledgment signal receiving unit 308 
receives positive acknowledgment (ACK) signals from 
the moving-picture decoder, and notifies the reference 
frame updating unit 309 of frames that have been posi- 
tively acknowledged. In multi-point transmission, the 

30 acknowledgment signal receiving unit 308 notifies the 
reference frame updating unit 309 of frames that have 
been positively acknowledged by at least a certain 
number of receiving sites, preferably by all receiving 
sites. 

35 The reference frame updating unit 309 updates the 
reference frame memory unit 305, responsive to the 
notifications received from the intra/inter decision unit 
306 and acknowledgment signal receiving unit 308. 
Upon receiving an intra-frame coding notification from 

40 the intra/inter decision unit 306, the reference frame 
updating unit 309 waits for the current frame to be 
coded by the coding unit 302, decoded by the decoding 
unit 303, and written in the frame memory unit 304, then 
copies this frame from the frame memory unit 304 to the 

45 reference frame memory unit 305, and deletes all frame 
data from the frame memory unit 304. Upon receiving 
the frame number of a positively acknowledged frame 
from the acknowledgment signal receiving unit 308, the 
reference frame updating unit 309 waits for the coding 

50 unit 302 to finish using the frame stored in the reference 
frame memory unit 305 to code the current frame, then 
copies the positively acknowledged frame from the 
frame memory unit 304 to the reference frame memory 
unit 305, and deletes frame data up to and including the 

55 positively acknowledged frame from the frame memory 
unit 304. 

FIG. 2 is a functional block diagram of the moving- 
picture decoder 400 in the first embodiment. This 
decoder, which is disposed in the receiving device, 
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comprises a coded data input unit 401, a reference 
frame comparison unit 402, a reference frame updating 
unit 403, a reference frame memory unit 404, a frame 
memory unit 405, a decoding unit 406. an acknowledg- 
ment signal transmitting unit 407, a moving -picture out- 5 
put unit 408, and a refresh signal transmitting unit 409. 

The coded data input unit 401 receives the coded 
data and header information transmitted from the mov- 
ing-picture coder 300. and furnishes the coded data to 
the decoding unit 406. From the header information, the 10 
coded data input unit 401 extracts the intra/inter flag and 
(for P-frames) the reference frame number, furnishes 
the intra/inter flag to the decoding unit 406, and fur- 
nishes the reference frame number to both the decoding 
unit 406 and the reference frame comparison unit 402. 15 

The reference frame comparison unit 402 com- 
pares the reference frame number received from the 
coded data input unit 401 with a reference frame 
number stored in the reference frame memory unit 404. 
If the received reference frame number is newer than 20 
the stored reference frame number, the reference frame 
comparison unit 402 passes the received reference 
frame number to the reference frame updating unit 403. 
together with a reference frame update request. 

Upon receiving a reference frame update request. 25 
the reference frame updating unit 403 searches for the 
frame with the received reference frame number in the 
frame memory unit 405. If the frame is found, the refer- 
ence frame updating unit 403 copies this frame and its 
frame number into the reference frame memory unit 30 
404, thereby updating the reference frame. 

The reference frame memory unit 404 stores one 
reference frame and its frame number. The frame mem- 
ory unit 405 stores decoded frame data and frame num- 
bers of a number of most recent frames. Whenever the 35 
reference frame updating unit 403 updates the refer- 
ence frame by copying a frame from the frame memory 
unit 405 to the reference frame memory unit 404, the 
copied frame and all preceding frames are deleted from 
the frame memory unit 405. <o 

The decoding unit 406 decodes the coded data 
received from the coded data input unit 401 . with refer- 
ence to the attached intra/inter flag. When this flag indi- 
cates inter-frame coding, the decoding unit 406 first 
checks that the received reference frame number 45 
matches the reference frame number stored in the refer- 
ence frame memory unit 404, and decodes the data 
only if these two frame numbers match; the decoding in 
this case is inter-frame, with reference to the reference 
frame stored in the reference frame memory unit 404. so 

During the decoding process, the decoding unit 406 
performs an error check such as a cyclic redundancy 
check. If this check passes, the frame is considered to 
have been successfully decoded, and the decoding unit 
406 sends the decoded data to the moving-picture out- 55 
put unit 408. If the frame was an l-frame. the decoding 
unit 406 also writes the decoded data and frame 
number into the reference frame memory unit 404, thus 
updating the reference frame, and furnishes the frame 



number to the acknowledgment signal transmitting unit 
407. If the frame was a P-frame. the decoding unit 406 
writes the decoded data and frame number into the 
frame memory unit 405. Under certain conditions, the 
decoding unit 406 also furnishes the frame number of a 
successfully decoded P-frame to the acknowledgment 
signal transmitting unit 407. Various conditions can be 
used; in the following description, it will be assumed that 
the frame number is furnished to the acknowledgment 
signal transmitting unit 407 if the coded data exceed a 
certain threshold size. 

Upon receiving a frame number from the decoding 
unit 406, the acknowledgment signal transmitting unit 
407 transmits a positive acknowledgment (ACK) signal 
to the moving-picture coder 300. The ACK signal 
includes the frame number received from the decoding 
unit 406. 

The moving-picture output unit 408 outputs 
decoded data from the decoding unit 406 to a monitor 
display device or similar device. The refresh signal 
transmitting unit 409 receives refresh requests from a 
user-controlled input device 420 and sends correspond- 
ing refresh (RFSH) signals to the moving-picture coder 
300. 

Next, the operation of the first embodiment will be 
described with reference to FIGs. 3, 4. and 5. In these 
and subsequent drawings, acknowledgment signals are 
indicated by straight arrows, and reference relationships 
by curved arrows. 

FIG. 3 shows a case in which the coding and 
decoding processes and data transmission process are 
carried out at a comparatively high speed. The first 
frame (a) is coded as an l-frame. and is successfully 
transmitted to and decoded by the moving-picture 
decoder 400, which stores the decoded data in its refer- 
ence frame memory unit 404 and returns a positive 
acknowledgment signal to the moving-picture coder 
300, as indicated by the first arrow labeled ACK. 

Frame a is also decoded by the decoding unit 303 
at the moving-picture coder 300. The decoded data are 
written in the frame memory unit 304, then copied into 
the reference frame memory unit 305, so that the refer- 
ence frame memory units 305 and 404 in the coder and 
decoder contain identical decoded data for use as a ref- 
erence frame. 

The next few frames (b, c, d, and e) are coded and 
decoded successfully with reference to this reference 
frame (a). Frames b, c, and d do not differ greatly from 
frame a, so the coded data size is small and the moving- 
picture decoder 400 does not return an acknowledg- 
ment signal. Frame e differs sufficiently, however, to 
cause a certain increase in the size of the coded data; 
the moving-picture decoder 400 accordingly returns a 
positive acknowledgment signal for frame e. 

The moving-picture coder 300 receives this 
acknowledgment signal before coding of the next frame 
(f) begins. The reference frame updating unit 309 is thus 
able to copy the decoded data of frame e into the refer- 
ence frame memory unit 305 in time for frame f to be 
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coded with reference to frame e, thereby reducing the 
size of the coded data of frame f. 

When the moving-picture decoder 400 receives the 
coded data of frame f, the reference frame comparison 
unit 402 finds that the attached reference frame number 
(of frame e) is newer than the frame number (of frame a) 
stored in the reference frame memory unit 404, and 
directs the reference frame updating unit 403 to copy 
frame e from the frame memory unit 405 to the refer- 
ence frame memory unit 404. The decoding unit 406 
then successfully decodes frame f with reference to 
frame e. 

Frames g and h are also coded and successfully 
decoded with reference to frame e. 

In a multi-point connection, the reference frame is 
changed from frame a frame e only if at least a certain 
number of receiving sites, preferably all receiving sites, 
return positive acknowledgment signals for frame e. 

FIG. 4 illustrates a case in which the moving-picture 
decoder "returns a positive "acknowledgment signal for 
frame d, but because of slower coding, slower decod- 
ing, or a slower transmission speed, the moving -picture 
coder 300 receives this positive acknowledgment signal 
after the coding of frame e has already begun. Frame e 
is therefore coded and decoded with reference to the 
existing reference frame (a). The reference frame is 
then changed to frame d, and the next frames (f, g, and 
h) are coded and decoded with reference to frame d. 

FIG. 5 illustrates a case of frame dropout. Frames 
b, c, d, and e are coded with reference to the first I- 
frame (a). Frames b to d are decoded successfully, and 
a positive acknowledgment signal is returned for frame 
d. The coded data of frame e are dropped or damaged 
in transmission, as indicated by the X-mark in the draw- 
ing. As a result, the coded data either are not received, 
or are received but cannot be decoded successfully, as 
indicated by cross-hatching. This does not affect the fol- 
lowing frames t, g, and h, which are coded and decoded 
successfully with reference to the positively acknowl- 
edged frame d. The moving-picture decoder 400 does 
not have to wait for the next (-frame (i) in order to 
resume decoding after the dropout of frame e. 

If the user watching the moving picture output by 
the moving -picture output unit 408 notices undesirable 
picture deterioration, due to an undetected error, for 
example, he or she can press a button or perform a sim- 
ilar operation that sends a refresh command to the 
refresh signal transmitting unit 409. A refresh signal is 
then transmitted to the moving-picture coder 300, forc- 
ing the next frame to be coded as an l-frame, thereby 
restoring good picture quality. 

In the first embodiment, dropout of a P-f rame never 
prevents the decoding of subsequent frames, because 
only positively acknowledged P-frames can become ref- 
erence frames. Differing from the prior art of FIG. 23. 
decoding can continue without waiting for the arrival of 
the next l-frame. At the same time, when a P-frame is 
successfully decoded and positively acknowledged, it 
can be used as a reference frame for coding subse- 



quent frames, thereby reducing the temporal distance 
between the coded and reference frames, and improv- 
ing the data compression ratio as compared with the 
prior art shown in FIG. 26. 

5 In particular, when system conditions permit sub- 

stantially all frames to be transmitted and decoded suc- 
cessfully, if the coded data size threshold for sending an 
acknowledgment signal is set at a sufficiently low level, 
the data compression performance of the first embodi- 

io ment will approach that of the scheme illustrated in FIG. 
22, in which every frame is coded with reference to the 
preceding frame. 

When channel conditions are less favorable, or the 
threshold is higher, positive acknowledgment signals 

is will be less frequent, reference frame updates will take 
place less often, and the data compression ratio will be 
reduced accordingly The first embodiment still achieves 
a higher data compression ratio than in the prior-art 
scheme of FIG. 26, however, while providing the same 

20 protection against frame dropouts. 

The first embodiment thus allows the data com- 
pression to vary according to channel conditions, ena- 
bling the transmission system to take advantage of 
good channel conditions to achieve high compression 

25 ratios, while maintaining good picture quality under 
even adverse channel conditions. 

FIG. 6 illustrates a variation of the first embodiment 
in which the moving-picture coder 500 has no decoding 
unit. Other elements of the coder are the same as in 

30 FIG. 1, and have the same reference numerals. The 
operation of this variation is as described above, except 
that the moving-picture input unit 301 writes each 
received frame directly into the frame memory unit 304. 
Besides simplifying the configuration and operation of 

35 the moving-picture coder 500, this variation has advan- 
tages related to the accuracy of motion compensation. 
The disadvantage is that the moving-picture coder and 
moving-picture decoder do not code and decode P- 
frames with reference to exactly the same data. 

40 As another variation, the decoding unit 303 in FIG. 
1 or moving-picture input unit 301 in FIG. 6 can be 
adapted to write l-frame data directly into the reference 
frame memory unit 305, instead of having the data first 
written into the frame memory unit 304, then copied to 

45 the reference frame unit 305 by the reference frame 
updating unit 309. This arrangement simplifies the oper- 
ation of the moving-picture coder. 

As still another variation, the reference frame mem- 
ory unit 305 and frame memory unit 304 can be com- 

50 bined into a single frame memory unit, using a pointer to 
indicate which stored frame is the reference frame. By 
eliminating all copying of data from one memory unit to 
another, this arrangement further simplifies operation of 
the moving-picture coder, and reduces the memory 

55 requirements. 

This combined frame memory unit can be managed 
as a ring buffer, with new data being overwritten on old 
data when the memory becomes full, thereby eliminat- 
ing the need to delete old data. The same scheme can 
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be applied to the frame memory unit 304 when not com- 
bined with the reference frame memory unit 305. 
* As yet another variation, the decoding unit 406 in 

the moving-picture decoder 400 in FIG. 2 can be 
adapted to evaluate the decoding process, and issue a 
refresh command to the refresh signal transmitting unit 
409 whenever symptoms of poor picture quality or other 
decoding problems appear. The refresh signal transmit- 
ting unit 409 can then transmit refresh signals without 
requiring input from the user. A refresh signal can also 
be transmitted whenever an l-frame cannot be success- 
fully decoded. 

In a further variation, the moving -picture coder 300 
divides each frame into a number of blocks, and codes 
each block separately. In a P-frame, each block is coded 
with reference to the corresponding block in the refer- 
ence frame. A separate reference frame number can be 
maintained for each block, so that different blocks in the 
same frame can be coded with reference to different ref- 
erence frames, and a separate acknowledgment signal 
can be sent for each block. Since a block is smaller than 
a frame, the probability of error is reduced, the overall 
reference frame update rate can be improved, the data 
compression ratio can be improved, and the effect of a 
dropout can be limited to a particular block in a frame. 

Second embodiment 

The second embodiment is identical to the first 
embodiment except that the moving-picture decoder 
400 acknowledges every successfully decoded frame. 

FIG. 7 illustrates the internal structure of the refer- 
ence frame updating unit 309A, which comprises a 
frame number setting unit 310 and a data copying unit 
311. This reference frame updating unit 309A exempli- 
fies a preferred internal structure of the reference frame 
updating unit 309 in the first embodiment. 

The frame number setting unit 310 sets the frame 
numbers of frames for which positive acknowledgment 
is received by the acknowledgment signal receiving unit 
308. These frame numbers are set in an internal work 
memory area (not visible), and passed to the data cop- 
ying unit 311. 

The data copying unit 31 1 copies the frames with 
numbers received from the frame number setting unit 
310 from the frame memory unit 304 to the reference 
frame memory unit 305. 

Next, the operation of the second embodiment will 
be described. 

The reference frame updating unit 309A updates 
the reference frame according to the ACK signals trans- 
mitted from the moving -picture decoder 400. In FIG. 8. 
for example, the moving -picture decoder 400 acknowl- 
edges frames a. b, and c. as indicated by the arrows 
marked ACK. The moving-picture coder 300 receives 
the ACK signal for frame b after the coding of frame c 
has begun, so frame c is still coded with reference to 
frame a, but frame d is coded with reference to frame b. 
Similarly, frame e is coded with reference to frame c. 



The moving-picture decoder 400 decodes frames b and 
c with reference to frame a, frame d with reference to 
frame b, and frame e with reference to frame c. 

Decoding of frame d is unsuccessful, so the mov- 

5 ing-picture decoder 400 does not transmit an ACK sig- 
nal for frame d. The moving-picture coder 300 therefore 
does not change the reference frame when coding 
frame f, but continues to use frame c as the reference 
frame. The moving-picture decoder 400 is thus able to 

w decode frame f. 

Frame e was successfully decoded and acknowl- 
edged, so frame g is coded with reference to frame e. 
Frame g can also be successfully decoded. 

In the second embodiment, as in the first embodi- 

15 ment, the dropout of frame d has no effect on the decod- 
ing of subsequent frames, which can be decoded 
successfully without waiting for the next l-frame. The 
degradation of picture quality resulting from the dropout 
is minimized. The data compression ratio is also good, 

20 as frames c, d, e, g. and h are all coded with reference 
to the frame two frames before, and frame f is coded 
with reference to the frame only three frames before. 
The use of frame c instead of frame d as a reference 
frame for frame f will normally affect the data compres- 

25 sion ratio only slightly. 

FIG. 9 shows an example of a dropout on the chan- 
nel by which the ACK signal is transmitted from the 
moving-picture decoder 400 to the moving -pi dure 
coder 300. In this example, frame d is successfully 

30 decoded, but the corresponding ACK signal is lost en 
route. The moving-picture coder 300 reacts in the same 
way as in FIG. 8, by coding frame f with reference to 
frame c, frame g with reference to frame e, and so on. 
All of these frames can be successfully decoded, and 

35 the data compression ratio is the same as in FIG. 8. 

The second embodiment accordingly has a high tol- 
erance for dropouts of signals and data traveling in both 
directions between the coder and decoder. 

40 Third embodiment 

The third embodiment differs from the preceding 
embodiments in that the moving-picture decoder 400 
transmits negative acknowledgment (NACK) signals 

45 instead of positive acknowledgment signals. A negative 
acknowledgment signal is transmitted when a frame is 
dropped or uncorrectably damaged in transmission. 
The negative acknowledgment signal includes the 
frame number of the frame that was not received, or 

50 was received with uncorrectable errors. 

Aside from this difference, the moving-picture 
decoder 400 in the third embodiment operates in the 
same way as in the first embodiment, with the configu- 
ration shown in FIG. 2. 

55 The moving-picture coder 300 has the configuration 
shown in FIG. 1, but the operation of the reference 
frame updating unit differs. 

FIG. 10 illustrates the internal structure of the refer- 
ence frame updating unit 309B in the third embodiment, 
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the suffix B indicating that this structure differs from the 
structure in the second embodiment. The component 
elements are a frame number setting unit 31 OA, a data 
copying unit 311 A, and a frame number confirming unit 
312, the suffixes A also indicating differences from the 
second embodiment. 

The frame number setting unit 31 OA supplies the 
frame numbers of negatively acknowledged frames, 
received from the acknowledgment signal receiving unit 
308. to the frame number confirming unit 312, and 
receives from the frame number confirming unit 312 in 
return a most recent confirmed frame number. The 
frame number setting unit 31 OA sets each most recent 
confirmed frame number in an internal memory (not vis- 
ible), and passes these most recent confirmed frame 
numbers to the data copying unit 31 1 A. 

Upon receiving a negatively acknowledged frame 
number from the frame number setting unit 31 OA, the 
frame number confirming unit 312 searches among the 
frame numbersstored in the~frame memory unit 304 to 
find the most recent frame number that is older than the 
frame number received from the frame number setting 
unit 31 OA, and has not been negatively acknowledged. 
This frame number is returned to the frame number set- 
ting unit 31 OA as the most recent confirmed frame 
number. 

The data copying unit 311 A normally copies each 
frame from the frame memory unit 304 to the reference 
frame memory unit 305 as soon as that frame is written 
into the frame memory unit 304 by the decoding unit 
303 in FIG. 1 , before the coding unit 302 begins coding 
the next frame. Upon receiving a most recent confirmed 
frame number from the frame number setting unit 31 OA, 
however, the data copying unit 31 1 A copies the frame 
with that frame number from the frame memory unit 304 
to the reference frame memory unit 305. 

Other parts of the moving-picture coder 300 oper- 
ate as in the first embodiment, except that frame data 
are not deleted from the frame memory unit 304 when- 
ever a new reference frame is copied into the reference 
frame memory unit 305. In the absence of a negative 
acknowledgment signal, frame data are retained in the 
frame memory unit 304 for a certain length of time, pref- 
erably as long as possible. 

Next, the operation of the third embodiment will be 
described with reference to FIG. 11. 

As long as the moving-picture coder 300 does not 
receive a negative acknowledgment signal, the refer- 
ence frame updating unit 309B assumes that all trans- 
mitted frames have been decoded successfully, and 
keeps updating the reference frame to the most recently 
coded frame. Normally, therefore, every frame is coded 
with reference to the immediately preceding frame, as 
indicated by the solid arrows at the top of FIG. 11. 
Frames b. c. d. and e in FIG. 1 1 are coded in this way. 

Frames a, b, and c are decoded successfully, but 
frame d is dropped or damaged in transmission. The 
moving-picture decoder 400 returns a negative 
acknowledgment signal (NACK). This NACK signal 



arrives at the moving-picture coder 300 while the coding 
unit 302 is coding frame e with reference to frame d. As 
soon as the coding unit 302 has finished referring to 
frame d, the reference frame updating unit 309B 

5 updates the reference frame by copying frame c, the 
most recent frame before frame d that has not been 
negatively acknowledged, from the frame memory unit 
304 to the reference frame memory unit 305. Frames d 
and e can be deleted from the frame memory unit 304 

10 at this point, because it is known that the moving-picture 
decoder 400 was unable to decode frame d and will be 
unable to decode frame e. 

Frame f is now coded with reference to frame c, as 
indicated by the dotted arrow at the top ol FIG. 1 1 . The 

15 moving-picture decoder 400 is therefore able to decode 
frame f. 

Frame e cannot be decoded because it was coded 
with reference to frame d, but frame e itself is not 
dropped or^ damaged in transmission^ Accordingly,. no 

20 negative^ acknowledgment signal is returned by the 
moving-picture decoder 400 for frame e while the mov- 
ing-picture coder 300 is coding frame f. 

The data copying unit 31 1 B therefore copies frame 
f from the frame memory unit 304 to the reference frame 

25 memory unit 305. and frame g is coded with reference 
to frame f. Similarly, each of frames h. i. j, and k is coded 
with reference to the immediately preceding frame. 

Compared with the second embodiment, the third 
embodiment normally achieves a higher data compres- 

30 sion ratio, because each frame is coded with reference 
to the immediately preceding frame, instead of the 
frame two frames before. When a frame dropout occurs, 
one or more following frames may become undecodable 
(e.g. frame e in FIG. 11), but decoding will quickly 

35 become possible again without the need to wait for the 
next l-frame. 

. In multi-point transmission, the acknowledgment 
signal receiving unit 308 reports a frame as having been 
negatively acknowledged if it is negatively acknowl- 
40 edged by at least a certain number of receiving sites. 
For example, a frame can be reported as negatively 
acknowledged if it is negatively acknowledged by at 
least one site. 



* The fourth embodiment is generally similar to the 
third embodiment, but the moving-picture decoder 400 
transmits negative acknowledgment (NACK) signals for 
all frames that could not be decoded successfully, 
regardless of whether the cause was a transmission 
error or dropout in the frame itself, or a transmission 
error or dropout in a preceding frame. This scheme is 
preferable when the possibility of a dropout on the 
acknowledgment signal channel exists. Each NACK sig- 
nal specifies both the frame number of the frame that 
could not be decoded, and a desired reference frame 
number. The desired reference frame number is the 
frame number of the most recent frame that was suc- 
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cessfulty decoded. 

The following description will be confined to the 
** structure and operation of the reference frame updating 
unit in the moving-picture coder. 

FIG. 12 illustrates the internal structure of the refer- 
ence frame updating unit 309C, the suffix C indicating 
that the structure differs from the third embodiment. The 
component elements are a frame number setting unit 
310C, a data copying unit 311 B, a frame number con- 
firming unit 31 2A, a candidate number deleting unit 31 3, 
and a candidate number list-keeping unit 314. 

The frame number setting unit 31 OC passes each 
negatively acknowledged frame number to the frame 
number confirming unit 312A. If the frame number con- 
firming unit 312A confirms the negatively acknowledged 
frame number as described below, the frame number 
setting unit 310C passes the desired reference frame 
number to the data copying unit 31 1B and candidate 
number deleting unit 313. If the negatively acknowl- 
edged frame number is not confirmed, the frame 
number setting unit 310C takes no action. 

Upon receiving a negatively acknowledged frame 
number from the frame number setting unit 31 0C, the 
frame number confirming unit 312A searches for the 
negatively acknowledged frame number in a candidate 
number list stored in the candidate number list-keeping 
unit 314, and notifies the frame number setting unit 
310C as to whether the negatively acknowledged frame 
number is present on the candidate number list. A frame 
number reported to be present on the candidate 
number list is considered to have been confirmed. 

Upon receiving a desired reference frame number, 
the candidate number deleting unit 313 deletes all 
frame numbers more recent than the desired reference 
frame number from the candidate number list. When a 
frame number is deleted from the candidate number list, 
the corresponding frame data can also be deleted from 
the frame memory unit 304. 

The candidate number list-keeping unit 314 stores 
and updates the candidate number list. As each frame 
is coded by the coding unit 302, its frame number is 
added to the candidate number list. When a frame has 
been deleted from both the frame memory unit 304 and 
reference frame memory unit 305, its frame number is 
removed from the candidate number list. 

The data copying unit 31 1B operates like the data 
copying unit 311 A in the third embodiment, normally 
copying each frame from the frame memory unit 304 to 
the reference frame memory unit 305 as soon as the 
decoding unit 303 has decoded the frame. Upon receiv- 
ing a desired reference frame number from the frame 
number setting unit 31 0C. however, the data copying 
unit 31 1 B copies the frame with that frame number from 
the frame memory unit 304 to the reference frame mem- 
ory unit 305. 

Next, the operation of the fourth embodiment will be 
described with reference to FIGs. 13 and 14. 

FIG. 13 illustrates the same situation as FIG. 1 1 , in 
which frame d is lost in transmission, frames d and e 



cannot be decoded, and NACK signals are returned for 
these two frames d and e. Both NACK signals specify 
the frame number of frame c as a desired reference 
frame number. 

5 Until these NACK signals arrive, the reference 

frame updating unit 309C assumes that all frames have 
been successfully decoded, and updates the reference 
frame at every frame. Frames b to e are therefore all 
coded with reference to the immediately preceding 

10 frame. 

When the reference frame updating unit 309C 
receives the negative acknowledgment of frame d, 
frames a to d have been coded and transmitted and are 
already on the candidate number list, and frame e is 

15 currently being coded. The frame number confirming 
unit 312A confirms that frame d is on the candidate 
number list, so the frame number setting unit 31 0C 
passes the frame number of frame c to the data copying 
unit 31 1B and candidate number deleting unit 313. 

20 When the coding of frame e is completed, the data cop- 
ying unit 311 copies frame c from the frame memory 
unit 304 to the reference frame memory unit 305, and 
the candidate number deleting unit 313 deletes the 
frame numbers of frames d and e from the candidate 

25 number list. Frame d is deleted from the frame memory 
unit 304. Frame e is preferably not written into the frame 
memory unit 304; if written, however, it is immediately 
deleted. 

Frame f is coded with reference to frame c. During 

30 the coding of frame f, the reference frame updating unit 
309C receives a negative acknowledgment for frame e, 
which could not be decoded because it was coded with 
reference to the missing frame d. The frame number 
confirming unit 31 2 A searches for frame e on the candi- 

35 date list, but cannot find it because it has been deleted 
by the candidate number deleting unit 313 as noted 
above. Since the negatively acknowledged frame e is 
not confirmed by the frame number confirming unit 
312A, the frame number setting unit 310C does not sup- 

40 ply any frame numbers to the data copying unit 31 IB 
and the candidate number deleting unit 313. When 
frame f has been coded it is transmitted to the moving- 
picture decoder 400 and added to the candidate 
number list, and the data copying unit 31 1B copies 

45 frame f immediately into the reference frame memory 
305. 

Frame g is therefore coded with reference to the 
immediately preceding frame f. The coding of subse- 
quent frames continues in this manner. 

so FIG. 14 illustrates a case in which a negative 
acknowledgment signal is transmitted but fails to arrive. 
As before, frame d is dropped or damaged in transmis- 
sion, making frames d and e undecodable. This time, 
however, the NACK signal for frame d is also lost in 

55 transmission, so the reference frame updating unit 
309C fails to realize that frame d and could not be 
decoded, and continues with processing as usual, cod- 
ing frame f with reference to frame e. Frames a to e are 
all placed on the candidate number list. 



9 



BNSDOCID' <EP 07 639*4 A 2 I > 



17 



EP 0 763 944 A2 



18 



The moving-picture decoder 400 transmits another 
NACK signal for frame e, accompanied by the frame 
number of frame c as a desired reference frame 
number, and this NACK signal is received at the mov- 
ing-picture coder 300. The data copying unit 31 1 B cop- 
ies frame c from the frame memory unit 304 into the 
reference frame memory unit 305, and the candidate 
number deleting unit 313 deletes all frame numbers 
more recent than frame c (frame numbers d, e, and f) 
from the candidate number list. Frame g is coded with 
reference to frame c. 

The moving-picture decoder 400 transmits a further 
NACK signal for frame f, again accompanied by frame 
number c as a desired reference frame number. Since 
frame number f has been deleted from the candidate 
number list, however, the reference frame updating unit 
309C ignores this NACK signal, and copies frame g, 
rather than frame c, from the frame memory unit 304 to 
the reference frame memory unit 305. Frame h can thus 
be coded with reference to frame g. 

Due to the loss of both frame d and its negative 
acknowledgment signal, three frames (d ( e, and f) can- 
not be decoded, but the system recovers nevertheless. 
Frame g and the subsequent frames can all be decoded 
successfully, so a lengthy episode of picture degrada- 
tion is avoided. Moreover, all frames except frame g are 
coded with reference to the immediately preceding 
frame, so a high data compression ratio is maintained. 

The difference between the third and fourth embod- 
iments is that by having the frame number confirming 
unit 312 check whether negatively acknowledged 
frames are present on the candidate number list, and 
ignoring the NACK signal if the negatively acknowl- 
edged frame number is not on the candidate number 
list, the fourth embodiment avoids unnecessarily retain- 
ing an old reference frame in response to repeated 
NACK signals. In FIG. 13, that is, frame g is coded with 
reference to frame c. whereas in the third embodiment 
(FIG. 11), frame g was coded with reference to frame c. 
When a frame dropout occurs, the fourth embodiment 
can accordingly achieve a better data compression ratio 
than can the third embodiment. This advantage of the 
fourth embodiment becomes increasingly important 
with increasing round-trip transmission time between 
the moving -picture coder 300 and decoder 400. 

As a variation of the fourth embodiment, the mov- 
ing-picture decoder 400 can omit the desired reference 
frame numbers from the NACK signals, and the moving- 
picture coder 300 can treat the last preceding frame that 
was not negatively acknowledged as the desired refer- 
ence frame This variation is suitable when the NACK 
signal transmission channel is highly reliable. 

As another variation, the moving-picture decoder 
400 can transmit both ACK and NACK signals, and the 
reference frame updating unit 309C in the moving-pic- 
ture coder 300 can treat the most recent frame number 
for which an ACK signal was received as the desired ref- 
erence frame number. 



Fifth embodiment 

The transmission system in the fifth embodiment 
sends a positive or negative acknowledgment signal for 

5 every frame, and adjusts the reference frame update 
mode adaptively, in response to channel conditions. 

Under favorable channel conditions, the reference 
frame number is normally updated at every frame, but is 
set back when a negative acknowledgment is received, 

10 as in the third and fourth embodiments. This mode of 
operation will be referred to as NACK mode. 

Under unfavorable channel conditions, the refer- 
ence frame number is updated only when a positive 
acknowledgment is received, as in the first and second 

15 embodiments. This mode of operation will be referred to 
as ACK mode. 

The moving-picture coder accordingly has a chan- 
nel assessing unit and a mode switching unit, in addition 
to the reference frame updating unit. . _ . .. 

20 FIG. 15 shows the configuration of the moving-pic- 

ture coder 600 in the fifth embodiment, using the same 
reference numerals as in FIG. 1 for identical elements, 
and reference numerals with letter suffixes for similar 
elements. The following description will focus on the 

25 elements which differ from or were not present in the 
first embodiment. These elements include the acknowl- 
edgment signal receiving unit 308A, reference frame 
updating unit 309D, channel assessing unit 315, and 
update mode switching unit 316. 

30 The acknowledgment signal receiving unit 308 A 
receives acknowledgment signals, and passes the pos- 
itive or negative acknowledgment information to the ref- 
erence frame updating unit 309D and channel 
assessing unit 315. If an acknowledgment signal fails to 

35 arrive within a certain time, the acknowledgment signal 
receiving unit 308A regards the relevant frame as hav- 
ing been negatively acknowledged. 

In multi-point transmission, the acknowledgment 
signal receiving unit 308 A receives acknowledgment 

40 signals from each destination site, calculates the frame 
numbers of the frames that were positively acknowl- 
edged by at least a certain number of sites (preferably 
all sites, and treats other frame numbers as having been 
negatively acknowledged. 

45 The reference frame updating unit 309D updates 
the reference frame responsive to notifications received 
from the intra/inter decision unit 306 and acknowledg- 
ment signal receiving unit 308A, in the update mode 
designated by the update mode switching unit 316. 

so In ACK mode, upon receiving positive acknowledg- 
ment for a particular frame number from the acknowl- 
edgment signal receiving unit 308A, the reference 
frame updating unit 309D copies the corresponding 
frame data from the frame memory unit 304 to the refer- 

55 ence frame memory unit 305. Data preceding that 
number may be deleted from the frame memory unit 
304 at this time. No action is taken in response to a neg- 
ative acknowledgment. 

In NACK mode, the reference frame updating unit 
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309D normally copies each frame from the frame mem- 
ory unit 304 to the reference frame memory unit 305 just 
before the coding unit 302 begins coding the next frame. 
When a negative acknowledgment is received from the 
acknowledgment signal receiving unit 308A, however, 
the most recent positively acknowledged frame is cop- 
ied from the frame memory unit 304 to the reference 
frame memory unit 305, thus setting the reference 
frame back. Information identifying the most recent pos- 
itively acknowledged frame is maintained in the refer- 
ence frame updating unit 309D and updated whenever 
a positive acknowledgment is received from the 
acknowledgment signal receiving unit 308A. Frames 
with frame numbers older than the most recent posi- 
tively acknowledged frame number may be deleted from 
the frame memory unit 304 at this time. 

Upon receiving an intra-frame coding notification 
from the intra/inter decision unit 306, the reference 
frame updating unit 309 copies the data of the current 
frame (l-frame) from the frame memory unit 304 to the 
reference frame memory unit 305, and deletes all frame 
data from the frame memory unit 304. 

The channel assessing unit 315 assesses channel 
quality according to the acknowledgment signals 
received from the acknowledgment signal receiving unit 
308 A, and notifies the update mode switching unit 316 
of the result of the assessment In the following descrip- 
tion there will be only two assessment levels: good and 
bad. Channel quality is assessed as bad if, for example, 
M or more of the N most recent frames were negatively 
acknowledged, and as good if fewer than M of these N 
frames were negatively acknowledged (M and N being 
positive integers such that N > M). 

Other assessment criteria can be used. For exam- 
ple, channel quality can be assessed as bad when L 
consecutive frames are negatively acknowledged, 
where L is a suitable positive integer. 

The update mode switching unit 316 switches the 
update mode according to the assessment made by the 
channel assessing unit 315. The mode can be switched 
to NACK mode when the assessment result is good for 
a first number of consecutive frames, for example, and 
to ACK mode when the assessment result is bad for a 
second number of consecutive frames. 

FIG. 16 illustrates the configuration of the moving- 
picture decoder 700 in the fifth embodiment, using the 
same reference numerals as in FIG. 2 for identical ele- 
ments, and reference numerals with letter suffixes for 
similar elements. The configuration of the moving-pic- 
ture decoder of the present embodiment is basically the 
same as in the first embodiment. The functions and 
operation of some of the elements differ, however; these 
differences will be described below. 

The reference frame comparison unit 402A com- 
pares the reference frame number received with each 
frame and the reference frame number recorded in the 
reference frame memory unit 404, and issues a refer- 
ence frame update request to the reference frame 
updating unit 403 if the reference frame numbers differ. 



Whereas the first embodiment only updated the refer- 
ence frame number in the forward direction, to a newer 
frame number, the fifth embodiment also allows the ref- 
erence frame number to be updated in the backward 

5 direction, to an older frame number. 

The decoding unit 406A decodes the received 
frame data as in the first embodiment, writing decoded 
l-frames into the reference frame memory unit 404, and 
decoded P-frames into the frame memory unit 405. The 

w decoding unit 406A also checks for decoding errors, 
using a cyclic redundancy check, for example, and 
informs the acknowledgment signal transmitting unit 
407 whether each frame was successfully or unsuc- 
cessfully received. 

is Next, the operation of the fifth embodiment will be 
described. The description will focus on operations in 
the moving-picture coder 300. which differ depending 
on whether the ACK mode or NACK mode is selected. 
To simplify the description, channel quality will be 

20 assessed as good whenever a positive acknowledg- 
ment signal is received, and as bad whenever a nega- 
tive acknowledgment signal is received. The operating 
mode is switched from NACK to ACK when the channel 
quality is assessed as bad for even one frame, and from 

25 ACK to NACK when the channel quality is assessed as 
good for four consecutive frames. 

Referring to FIG. 17, the moving-picture coder 300 
initially operates in the NACK mode, coding each frame 
with reference to the preceding frame. Frames a, b ( and 

30 c are successfully decoded, but a dropout occurs at 
frame d. When the NACK signal, indicated by a dot- 
dash arrow, reporting this dropout is received, it is too 
late to prevent frame e from being coded with reference 
to frame d, but for frame f, the reference frame number 

35 is set back to frame c, the most recent frame that has 
been positively acknowledged. Frame f is thus coded 
with reference to frame c. 

The moving-picture decoder 400 cannot decode 
frame e, but can decode frame f. The moving-picture 

40 decoder 400 therefore sends a negative acknowledg- 
ment, indicated in FIG. 17 by another dot-dash arrow, 
for frame e, and a positive acknowledgment, indicated 
by a solid arrow, for frame f. 

In the meantime, the moving-picture coder 300 has 

45 switched to ACK mode, because of the negative 
acknowledgment of frame d. The NACK signal for frame 
e is therefore ignored. The reference frame is left 
unchanged, and frame g is coded with reference to 
frame c. 

so While frame g is being coded, the positive acknowl- 
edgment signal for frame f is received. The next frame h 
is therefore coded with reference to frame f. Similarly, 
the positive acknowledgment signal received for frame g 
causes frame i to be coded with reference to frame g. 

55 Another dropout occurs at frame h. The NACK sig- 
nal for frame h is received before the coding of frame j. 
so the reference frame number is not updated when 
frame j is coded. Frame j is coded, and successfully 
decoded, with reference to frame g. During the coding 
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of frame j, an ACK signal is received for frame i, so the 
next frame k is coded with reference to frame i. 

A further dropout occurs at frame k. The resulting 
NACK signal causes frame m to be coded, and suc- 
cessfully decoded, with reference to frame j, while the s 
positive acknowledgment of frame I causes frame n to 
be coded, and successfully decoded, with reference to 
frame I. 

Frames I. m, n, and o are all successfully decoded. 
Their positive acknowledgment signals cause the mov- w 
ing-picture coder 300 to switch back to NACK mode 
after coding frame q. Frame r is therefore coded with 
reference to frame q, and frame s with reference to 
frame r. 

Under good channel conditions, when there are no 15 
dropouts, the reference frame number is updated at 
every frame, so a good data compression ratio is 
achieved even if there is a large round-trip transmission 
delay. If an occasional dropout occurs, the next frame or 
frames rhay becorne undecodable, because the system 20 
is operating in NACK mode, but the resulting degrada- 
tion of picture quality will tend to be subjectively disre- 
garded, because the transmission is dominated by long 
dropout-free intervals with good picture quality. 

If channel conditions deteriorate, the moving-pic- 25 
ture coder 300 switches from NACK to ACK mode and 
begins using only positively acknowledged frames as 
reference frames, so that the dropout of one P-frame 
does not prevent other frames from being decoded. In 
this mode, the system can tolerate consecutive drop- 30 
outs without having such dropouts lead to long runs of 
undecodable frames. Picture degradation is thus limited 
to the unavoidable degradation caused by the dropouts 
themselves. Under poor channel conditions, the fifth 
embodiment offers a picture-quality improvement of 35 
about one to five decibels over the third and fourth 
embodiments, to say nothing of the prior art in FIG. 22. 

Following a dropout in either ACK or NACK mode, 
successful decoding of P -frames can resume without 
waiting for the next l-frame. 40 

The data compression ratio adapts sensitively to 
channel conditions. As channel quality decreases, the 
moving-picture coder 300 operates for an increasing 
proportion of the time in ACK mode, and the data com- 
pression ratio decreases accordingly, but as long as any 45 
time is spent in NACK mode, the data compression ratio 
will be better than in the first or second embodiment. 

These advantages of the fifth embodiment are 
achieved by using the existing acknowledgment signals 
to assess channel quality. There is, accordingly, no so 
need to transmit new information expressly for the pur- 
pose of channel assessment. 

Another advantage of the fifth embodiment is that 
operations on the receiving side do not depend on the 
reference frame updating mode. Only the moving-pic- 55 
ture coder 300 has to switch modes, and there is no 
need to transmit a mode notification signal to the receiv- 
ing side. Regarding this last point, if a mode notification 
signal were to be transmitted, strong error-detection 



and correction measures would have to be taken to 
ensure that the signal was received. The fifth embodi- 
ment thus saves not only the channel bandwidth - 
needed to transmit a mode notification signal, but also 
the bandwidth that would be needed for error protection. 

Next, several variations of the fifth embodiment will 
be described. 

In one variation, each frame is divided into blocks, 
and a separate reference frame number is provided for 
each block. This scheme has already been described 
as a variation of the first embodiment. If this scheme is 
employed in the fifth embodiment, the channel can be 
assessed on the basis of acknowledgment signals 
received for blocks in the same position in a series of 
consecutive frames. 

In another variation, in multi -point transmission, the 
acknowledgment signal receiving unit 308A passes the 
individual acknowledgment signals received from differ- 
ent destination sites toJhe channel-assessing unit 315, - 

instead of simply notifying the channel assessing unit 
315 as to whether a frame was decoded successfully at 
all sites or not. This variation enables the channel 
assessing unit 315 to assess channel conditions more 
sensitively. 

In still another variation, more than two channel 
assessment levels are provided, with a separate operat- 
ing mode for each level. A very bad level, for example, 
can be added to the good and bad levels described 
above. When the very bad level is recognized, the quan- 
tization step size in the coding process can be 
increased, to reduce the amount of coded data per 
frame and raise the percentage of frames that can be 
decoded successfully. 

In yet another variation, the moving-picture decoder 
400 transmits only ACK signals. Failure to receive an 
ACK signal within a designated time is regarded as neg- 
ative acknowledgment. This variation is suitable for 
transmission channels with a predictable round-trip 
transmission time. 

In still another variation, mode information is added 
as flag information to the data transmitted by the mov- 
ing-picture coder 300. The moving -picture decoder 400 
can use this information to delete unnecessary data 
from the frame memory unit 405. In the ACK mode, for 
example, data older than the current reference frame 
can be deleted. 

In yet another variation, the moving-picture decoder 
400 decides when to switch modes. From the acknowl- 
edgment signals, the moving-picture coder 300 can 
determine only whether each frame (or block) had an 
uncorrectable error, but the moving -picture decoder 400 
can determine the rate of correctable errors, and there- 
fore evaluate channel conditions more sensitively. 

In still another variation, l-frames are written by the 
decoding unit 303 directly into the reference frame 
memory unit 305, instead of being copied from the 
frame memory unit 304 to the reference frame memory 
unit 305 by the reference frame updating unit 309D. 

In yet another variation, the reference frame mem- 
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ory unit 305 and frame memory unit 304 are combined 
into a single memory unit, in which the reference frame 
is managed by a pointer. 

In still another variation, the channel assessing unit 

315 has a plurality of criteria for assessing the state of 
the channel. The channel state is assessed as good 
when it is good by all of these criteria, and as bad when 
it is bad by any one (or more) of the criteria. Alterna- 
tively, the state may be assessed as good when it is 
good by any one of the criteria, and bad when it is bad 
by all of the criteria, or the assessment may be changed 
only when the change is indicated by at least a certain 
number of the criteria 

Sixth embodiment 

The sixth embodiment is similar to the fifth embodi- 
ment, but also enables mode -switching to be controlled 
by the person operating the moving -picture coder. The 
configuration of the coder thus differs slightly from the 
fifth embodiment. The configuration and operation of 
the moving-picture decoder are the same as in the fifth 
embodiment, and will not be described 

FIG. 18 is a block diagram of the moving-picture 
coder 800 in the sixth embodiment, using the same ref- 
erence numerals for elements that are identical or 
equivalent to elements in the fifth embodiment (FIG. 
15). Descriptions of these elements will be omitted. 

The criteria modification unit 31 7 receives a criteria 
modification command from the human operator, deter- 
mines the appropriate new channel assessment criteria 
from this command, and notifies the channel assessing 
unit 315A. 

The channel assessing unit 315A. in addition to 
having the functions of the channel assessing unit 315 
described in the fifth embodiment, is adapted to change 
the channel assessment criteria to the criteria furnished 
by the criteria modification unit 317. 

The mode modification unit 318 receives a mode 
modification command from the human operator, and 
commands the update mode switching unit 31 6A to 
switch to the designated mode. 

The update mode switching unit 31 6A. in addition to 
having the functions of the update mode switching unit 

316 described in the fifth embodiment, is adapted to 
switch modes in response to commands received from 
the mode modification unit 318. The reference frame 
updating unit 309D then operates in the commanded 
mode until, for example, a reset signal is received, 
whereupon the update mode switching unit 31 6A 
resumes switching the update mode in response to 
channel conditions. 

Next, the operation of the sixth embodiment will be 
described. The description will be confined to opera- 
tions performed in response to input from the human 
operator of the moving-picture coder 800. Other opera- 
tions take place as in the fifth embodiment. 

The human operator of the moving -picture coder 
800 can control the reference frame updating process 



through two types of operations. 

One type of operation changes the criteria by which 
the state of the transmission channel is assessed. An 
example of this type of operation changes the criteria for 

5 a bad assessment from, for instance, negative acknowl- 
edgment of at least three of the last ten frames to nega- 
tive acknowledgment of at least seven of the last twenty 
frames. In this way the operator can influence the refer- 
ence frame updating mode while still allowing the mov- 

10 ing-picture coder to adapt to changing channel 
conditions. 

The other type of operation forces the moving-pic- 
ture coder 800 to operate in a designated mode, e.g. 
ACK mode or NACK mode, regardless of channel con- 
15 ditions. 

In addition to providing the benefits of the fifth 
embodiment, the sixth embodiment gives the human 
operator the option of exercising direct or indirect con- 
trol over the reference frame updating mode, hence 

20 over picture quality and the data compression ratio. This 
allows the operator to take action according to personal 
preferences, or according to specific channel condi- 
tions. The sixth embodiment is also useful in testing and 
evaluating different assessment criteria and reference 

25 frame updating modes. 

Seventh embodiment 

The seventh embodiment is also similar to the fifth 

30 embodiment, but gives the human user of the moving- 
picture decoder the options enjoyed by the operator of 
the moving-picture coder in the sixth embodiment. Both 
the moving-picture coder and moving-picture decoder 
differ slightly from the fifth embodiment. 

35 FIG. 19 illustrates the configuration of the moving- 
picture coder 900, using the same reference numerals 
as in FIG. 18 for elements that are the same as in the 
sixth embodiment. The following description will be con- 
fined to the differing elements. 

40 The criteria modification unit 31 7A is similar to the 
criteria modification unit 317 of the sixth embodiment, 
but receives criteria modification signals transmitted 
from the moving-picture decoder 1000, rather than com- 
mands input from an input device attached to the mov- 

45 ing-picture coder 900. 

The mode modification unit 318A is similar to the 
mode modification unit 31 8 of the sixth embodiment, but 
receives mode modification signals transmitted from the 
moving-picture decoder 1000, rather than commands 

so input from an input device attached to the moving-pic- 
ture coder 900. 

FIG. 20 illustrates the configuration of the moving- 
picture decoder 1000. using the same reference numer- 
als as in FIG. 16 for identical elements. The following 

55 description will be confined to the additional elements. 

The criteria modification signal transmitting unit 410 
receives criteria modification commands input by a 
human user through the input device 420, and transmits 
corresponding criteria modification signals to the mov- 
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ing-picture coder 900 via the transmission channel. 

The mode modification signal transmitting unit 41 1 , 
similarly, receives mode modification commands input 
by the user through the input device 420, and transmits 
corresponding mode modification signals to the moving- 5 
picture coder 900 via the transmission channel. 

Next, the operation of the seventh embodiment will 
be described. The description will be confined to the 
operations performed in response to commands input 
by the user of the moving-picture decoder 1000. w 

This user can perform three types of operations. 
One type of operation inputs a refresh command, as in 
the first embodiment. The other two types of operations 
input criteria modification commands and mode modifi- 
cation commands, these commands being similar to the is 
corresponding commands input by the operator of the 
moving-picture coder 900 in the sixth embodiment. 

Any one of these three operations causes the mov- 
ing-picture decoder 1000 to transmit a corresponding 
signal to the moving -picture" coder 900. A refresh signal 20 
forces the moving-picture coder 900 to code and trans- 
mit an l-frame. The criteria modification and mode mod- 
ification signals are processed as were the criteria 
modification and mode modification commands in the 
sixth embodiment. 25 

The seventh embodiment provides the same 
advantages as the fifth embodiment, with the additional 
advantage of giving the user of the moving-picture 
decoder 1 000 more control over picture quality. A partic- 
ular advantage of the seventh embodiment is that crite- 30 
ria modification commands and mode modification 
commands can be issued in response to the picture 
quality actually perceived by the user of the moving-pic- 
ture decoder 1000. 

As in the sixth embodiment, the ability to specify the 35 
channel assessment criteria and reference frame 
updating mode directly is useful in testing and evaluat- 
ing system performance under different criteria and 
modes. 

As a variation, the sixth and seventh embodiments 40 
can be combined, giving the human users at both ends 
the option of exercising control over the assessment cri- 
teria and reference frame update mode. 

Although the invention has been described as 
transmitting moving-picture data, it can be applied to 45 
any type of data coded by using both intra-frame and 
inter-frame coding methods. If a frame is divided into 
separately-coded blocks, the invented reference frame 
update methods can be applied separately to each 
block, as described above. The word "frame" can then so 
be interpreted to refer to one block, e.g. to one part of a 
picture. 

Numerous variations of the embodiments have 
been described above, but those skilled in the art will 
recognize that still further variations are possible within 55 
the scope of the invention. 



Claims 

1 . A coder for coding a series of frames by intra-frame 
coding and inter-frame coding, transmitting result- 
ing coded data to a decoder over a transmission 
channel, and receiving acknowledgment signals 
from said decoder, comprising: 

a memory means (304, 305) for storing a plu- 
rality of frames among said series of frames; 
a reference frame updating means (309) for 
selecting one frame stored in said memory 
means as a reference frame, responsive to 
said acknowledgment signals; 
a decision means (306) for designating which 
frames in said series of frames are to be coded 
by intra-frame coding, and which frames in said 
series of frames are to be coded by inter-frame 
coding; _ . _ _ 

a coding means (302) coupled to said memory 
means (304, 305) and said decision means 
(306), for performing intra-frame coding and 
inter-frame coding as designated by said deci- 
sion means and thereby generating coded 
data, inter-frame coding being performed with 
reference to the reference frame designated by 
said reference frame updating means (309); 
and 

a transmitting means (307) coupled to said 
coding means (302), for transmitting said 
coded data to said data decoder. 

2. The coder of claim 1 , wherein said series of frames 
constitute a moving picture. 

3. The coder of claim 1, wherein each frame in said 
series of frames represents one picture. 

4. The coder of claim 1 , wherein each frame in said 
series of frames represents one part of one picture. 

5. The coder of claim 1 , wherein said coder transmits 
said coded data to a plurality of decoders, and 
receives said acknowledgment signals from said 
plurality of decoders, and said reference frame 
updating means (309) selects, as said reference 
frame, a frame that has been positively acknowl- 
edged by at least a certain number of said decod- 
ers. 

6. The coder of claim 1 . wherein said coder transmits 
said coded data to a plurality of decoders, and 
receives said acknowledgment signals from said 
plurality of decoders, and said reference frame 
updating means (309) selects, as said reference 
frame, a frame that has not been negatively 
acknowledged by at least a certain number of said 
decoders. 
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7. The coder of claim 1, wherein said acknowledg- 
ment signals comprise positive acknowledgment 
signals designating successfully decoded frames, 
and when each one of said positive acknowledg- 
ment signals is received, said reference frame 
updating means (309) selects, as said reference 
frame, the frame designated by said one of said 
positive acknowledgment signals. 

8. The coder of claim 1, wherein said acknowledg- 
ment signals comprise negative acknowledgment 
signals designating unsuccessfully decoded 
frames, and when each one of said negative 
acknowledgment signals is received, said reference 
frame updating means (309) selects, as said refer- 
ence frame, a frame coded before the frame desig- 
nated by said one of said negative acknowledgment 
signals. 

9. The coder of claim 1, wherein said acknowledg- 
ment signals comprise negative acknowledgment 
signals designating unsuccessfully decoded frames 
and desired reference frames, and when each one 
of said negative acknowledgment signals is 
received, said reference frame updating means 
(309) selects, as said reference frame, the desired 
reference frame designated by said one of said 
negative acknowledgment signals. 

10. The coder of claim 1 , wherein said reference frame 
updating means (309) comprises: 

a list-keeping unit (314) for keeping a list of 
candidate reference frames, and adding the 
frames coded by said coding unit (302) to said 
list of candidate reference frames; and 
a deleting unit (313) for, when one of said 
acknowledgment signals designates a frame 
as having been unsuccessfully decoded, delet- 
ing the frame thus designated, and all more 
recent frames in said series of frames, from 
said list of candidate reference frames; 

and wherein said reference frame updating 
means (309) selects, as said reference frame, one 
of the frames on said list of candidate reference 
frames. 

11. The coder of claim 10, wherein said acknowledg- 
ment signals comprise negative acknowledgment 
signals designating both a frame that could not be 
decoded successfully and a desired reference 
frame. 

12. The coder of claim 11 wherein, if both said frame 
that could not be decoded successfully and said 
desired reference frame are present on said list of 
candidate reference frames, said reference frame 
updating means (309) selects said desired refer- 



ence frame as said reference frame. 

13. The coder of claim 11 wherein, when said frame 
that could not be decoded successfully is not on 
5 said list of candidate reference frames, said refer- 
ence frame updating means (309) selects a most 
recent frame on said list of candidate reference 
frames as said reference frame. 

io 14. The coder of claim 1 , also comprising: 

an assessing means (315) for assessing qual- 
ity of said transmission channel, based on said 
acknowledgment signals; and 
15 a mode switching means (316) coupled to said 

assessing means (315), for directing said refer- 
ence frame updating means (309) to select ref- 
erence frames in different modes, responsive 
to said quality. 

so 

15. The coder of claim 14, wherein: 

said assessing means (315) assesses said 
quality by counting negatively acknowledged 
frames among a certain number of most recently 
25 transmitted frames. 

16. The coder of claim 14, wherein said assessing 
means (315) assesses said quality by counting 
consecutive negatively acknowledged frames. 

30 

17. The coder of claim 14, wherein said frames repre- 
sent parts of pictures in a moving picture, and said 
assessing means (315) assesses said quality by 
counting negatively acknowledged frames in identi- 

35 cal positions in a certain number of consecutive pic- 
tures. 

18. The coder of claim 14, wherein said assessing 
means (315) assesses said quality according to a 

40 plurality of criteria. 

19. The coder of claim 14, wherein said assessing 
means (315) assesses said quality by making 
assessments of good and bad. 

45 

20. The coder of claim 19 wherein, when said quality is 
assessed as good, said mode switching means 
(316) directs said reference frame updating means 
(309) to select a most recently coded frame as said 

so reference frame, unless a negative acknowledg- 
ment signal is received. 

21. The coder of claim 19 wherein, when said quality is 
assessed as bad, said mode switching means 

55 (316) directs said reference frame updating means 
(309) to select, as said reference frame, only 
frames confirmed by said acknowledgment signals 
to have been decoded successfully. 



15 



BNSDOCID: <EP 07639A4A2 I > 



29 



EP 0 763 944 A2 



30 



22. The coder of claim 14, comprising a mode modify- 
ing means (318) for receiving a mode command 
from a human user, and forcing said reference 
frame updating means (309) to select said refer- 
ence frame in a mode designated by said mode 5 
command. 

23. The coder of claim 14, comprising a criteria modify- 
ing means (317) for receiving a criteria modification 
command from a human user, and modifying crite- 10 
ha by which said assessing means (315) assesses 
said quality, responsive to said criteria modification 
command. 

24. A decoder for receiving, through a transmission 15 
channel, coded data from a coder that performs 
intra-frame coding and inter-frame coding, and 
decoding said coded data to obtain a series of 
frames, comprising: 

20 

an acknowledgment signal transmitting means 
(407) for transmitting acknowledgment signals, 
acknowledging respective frames, to said 
coder. 

25 

25. The coder of claim 24. wherein said series of 
frames constitute a moving picture. 

26. The decoder of claim 24, wherein said acknowledg- 
ment signals comprise positive acknowledgment 3d? 
signals indicating that respective frames were suc- 
cessfully decoded. 

27. The decoder of claim 24, wherein said acknowledg- 
ment signals comprise negative acknowledgment 35 
signals indicating that respective frames were not 
successfully received. 

28. The decoder of claim 24, wherein said acknowledg- 
ment signals comprise negative acknowledgment 40 
signals indicating that respective frames were not 
successfully decoded. 

29. The decoder of claim 24. wherein said acknowledg- 
ment signals comprise positive acknowledgment 45 
signals indicating that respective frames were suc- 
cessfully decoded, and negative acknowledgment 
signals indicating that respective frames were not 
successfully received. 

50 

30. The decoder of claim 24. wherein said acknowledg- 
ment signals comprise negative acknowledgment 
signals indicating that respective frames were not 
successfully decoded, and each of said negative 
acknowledgment signals designates a desired ref- ss 
erence frame, to be used by said coder for inter- 
frame coding of following frames. 

31. The decoder of claim 24, wherein said coder 



assesses quality of said transmission channel 
according to certain criteria, and selects reference 
frames for use in inter-frame coding in a mode 
responsive to said quality, and said decoder also 
comprises: 

a criteria modification signal transmitting 
means (410) for receiving input from a human 
user, and transmitting to said coder a corre- 
sponding signal commanding said coder to 
modify said criteria, responsive to said input. 

32. The decoder of claim 24, wherein said coder 
assesses quality of said transmission channel 
according to certain criteria, and selects reference 
frames for use in inter-frame coding in a mode 
responsive to said quality, and said decoder also 
comprises: 



a mode modification signal transmitting means 
(411) for receiving input from a human user, 
and transmitting to said coder a signal com- 
manding said coder to modify said mode, 
responsive to said input. 

33. A method of transmitting a series of frames repre- 
senting a moving picture from a transmitting device 
to a receiving device, comprising the steps of: 

selecting a coding method for each frame in 
said series of frames, by deciding between 
intra-frame coding and inter-frame coding; 
coding frames selected for intra-frame coding 
without reference to other frames; 
coding frames selected for inter-frame coding 
with reference to a reference frame, said refer- 
ence frame being a frame that was coded pre- 
viously; 

transmitting coded data produced by the intra- 
frame coding and inter-frame coding of said 
frames from said transmitting device to said 
receiving device; 

decoding said coded data at said receiving 
device; 

transmitting acknowledgment signals from said 
receiving device to said transmitting device; 
and 

selecting said reference frame at said transmit- 
ting device, responsive to said acknowledg- 
ment signals. 

34. The method of claim 33, wherein each frame in said 
series of frames represents one picture. 

35. The method of claim 33. wherein each frame in said 
series of frames represents one part of one picture. 

36. The method of claim 33, wherein said coded data 
are transmitted to a plurality of receiving devices, 
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and said acknowledgment signals are received 
from said plurality of receiving devices. 

37. The method claim 36. wherein said step of select- 
ing said reference frame comprises: 

selecting a frame that has been positively 
acknowledged by at least a certain number of 
said receiving devices. 

38. The method claim 36. wherein said step of select- 
ing said reference frame comprises: 

selecting a frame that has not been negatively 
acknowledged by at least a certain number of 
said receiving devices. 

39. The method of claim 33, wherein said acknowledg- 
ment signals comprise positive acknowledgment 
signals designating successfully decoded frames, 
and said step of selecting said reference frame 
comprises: 

selecting a frame designated by one of said 
positive acknowledgment signals. 

40. The method of claim 33. wherein said acknowledg- 
ment signals comprise negative acknowledgment 
signals designating unsuccessfully decoded 
frames, and said step of selecting said reference 
frame comprises: 

selecting a frame coded before a frame desig- 
nated by one of said negative acknowledgment 
signals. 

41. The method of claim 33. wherein said acknowledg- 
ment signals comprise negative acknowledgment 
signals designating unsuccessfully decoded frames 
and desired reference frames, and said step of 
selecting said reference frame comprises: 

selecting said desired reference frames as said 
reference frame. 

42. The method of claim 33. wherein said step of 
selecting said reference frame comprises the fur- 
ther steps of: 

keeping a list of candidate reference frames; 
adding the frames coded by intra-frame coding 
and the frames coded by inter-frame coding to 
said list; 

deleting from said list each frame that is nega- 
tively acknowledged: 

simultaneously deleting from said list frames 
more recent than the frame that is negatively 
acknowledged; and 

selecting said reference frame from said list. 



43. The method of claim 42, wherein said acknowledg- 
ment signals comprise negative acknowledgment 
signals designating both a frame that could not be 
decoded successfully and a desired reference 

5 frame. 

44. The method of daim 43, wherein said step of 
selecting said reference frame comprises: 

w selecting said desired reference frame, if both 

said frame that could not be decoded success- 
fully and said desired reference frame are 
present on said list. 

is 45. The method of claim 43 wherein, wherein said step 
of selecting said reference frame comprises. 

selecting a most recent frame on said list, if 
said frame that could not be decoded success- 
20 fully is not on said list. 

46. The method of claim 33, comprising the further 
steps of: 

25 assessing quality of said transmission channel; 

and 

selecting said reference frames in different 
modes, responsive to said quality. 

30 47. The method of claim 46, wherein said transmitting 
device performs said step of assessing quality, 
based on said acknowledgment signals. 

48. The method of daim 47, wherein said step of 
35 assessing quality comprises: 

counting negatively acknowledged frames dur- 
ing a certain interval. 

40 49. The method of daim 47, wherein said step of 
assessing quality comprises: 

I counting consecutive negatively acknowledged 
frames. 

45 

50. The method of claim 47. wherein said frames repre- 
sent parts of pictures in said moving picture, and 
said step of assessing quality comprises: 

so counting negatively acknowledged frames in 

identical positions in a certain number of con- 
secutive pictures. 

51. The method of daim 46, wherein said step of 
55 assessing quality comprises making assessments 

of good and bad. 

52. The method of daim 51, wherein said step of 
selecting said reference frame comprises: 
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selecting a most recently coded frame, when 
said quality is assessed as good, unless a neg- 
ative acknowledgment signal is received. 

53. The method of claim 51, wherein said step of s 
selecting said reference frame comprises: 

selecting frames confirmed by said acknowl- 
edgment signals to have been decoded suc- 
cessfully, when said quality is assessed as w 
bad. 

54. The method of claim 46, comprising the further step 
of receiving a first command from a human user, 
specifying criteria by which said quality is to be is 
assessed. 

55. The method of claim 54, wherein said first com- 
mand is received from a human user at said trans- . 
rhitting device. 20 

56. The method of claim 54, wherein said first com- 
mand is received from a human user at said receiv- 
ing device. 

25 

57. The method of claim 46, comprising the further step 
of receiving a second command from a human 
user, specifying a mode of selecting said reference 
frame. 

30 

58. The method of claim 57, wherein said second com- 
mand is received from a human user at said trans- 
mitting device. 

59. The method of claim 57, wherein said second com- 35 
mand is received from a human user at said receiv- 
ing device, comprising the further step of: 

transmitting said second command from said 
receiving device to said transmitting device. 40 
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